1. Field of the Invention
The present invention relates to a technique for obtaining objective variables from multivariate data.
<Description of Symbols>
Symbols used in the following description will be defined below.
1. The elements of D-dimensional sample vectors x and y are denoted by x(1), x(2), . . . , x(D) and y(1), y(2), . . . , y(D), respectively.
2. The sample average values of N sample vectors x and y are denoted by <x> and <y>, respectively.
                              <          x          >                      :                          =                              (                          1              N                        )                    ⁢                                    (                                                                    ∑                                          j                      =                      1                                                              j                      =                      N                                                        ⁢                                      xj                    ⁡                                          (                      1                      )                                                                      ,                                                      ∑                                          j                      =                      1                                                              j                      =                      N                                                        ⁢                                      xj                    ⁡                                          (                      2                      )                                                                      ,                …                ⁢                                                                  ,                                                      ∑                                          j                      =                      1                                                              j                      =                      N                                                        ⁢                                      xj                    ⁡                                          (                      D                      )                                                                                  )                        T                                              (        1        )            
3. Variance-Covariance
The standard deviation Sxx of sample vectors xj is represented by an equation (2).
                    Sxx        =                              (                          1              N                        )                    ⁢                                    ∑                              j                =                1                                            j                =                N                                      ⁢                                                                                                xj                    -                                    <                  x                  >                                                            2                                                          (        2        )            
The standard deviation Syy of sample vectors yj is represented by an equation (3).
                    Syy        =                              (                          1              N                        )                    ⁢                                    ∑                              j                =                1                                            j                =                N                                      ⁢                                                                                                yj                    -                                    <                  y                  >                                                            2                                                          (        3        )            
Here, the covariance of x and y is represented by an equation (4).
                    Sxy        =                              (                          1              N                        )                    ⁢                                    ∑                              j                =                1                                            j                =                N                                      ⁢                                                                                                xj                    -                                    <                  x                  >                                                            ⁢                                                                                    yj                    -                                    <                  y                  >                                                                                                        (        4        )            
2. Description of the Related Art
In a case that N sets of an input vector x in an unknown system and an output vector y, or N measurement values of variables x and y are given and there is a linear relation between x and y, the relation can be written asY=θ1Tx+θ2  (5)
The relation between x and y can be obtained by obtaining parameters θ1 and θ2 in the equation (5). The technique for estimating the values of parameters θ1 and θ2 is known as the regression analysis technique.
As an exemplary application for regression analysis, multiple linear regression analysis for estimating the state of a process is disclosed in Japanese Patent Laid-Open No. 6-110504. Multiple linear regression analysis is also used in Japanese Patent Laid-Open No. 6-117932 for estimating the spectral reflectance of a minute sample from spectral reflectances measured with a calorimeter. In Japanese Patent Laid-Open No. 6-301669, multiple linear regression analysis is used to estimate the snowfall in an area where no snow accumulation measuring device is installed from snow accumulation information obtained from a snowfall accumulation measuring device with a high degree of accuracy. In Japanese Patent Laid-Open No. 6-350843, multiple linear regression analysis is used to estimate a reproduced color density from a primary color density and a reproduced color density measured from samples prepared by combining the three primary colors in various ways. Furthermore, in Japanese Patent Laid-Open No. 7-017346, multiple linear regression analysis is used to estimate a road friction coefficient from the braking pressure, wheel acceleration, and wheel slip ratio of a vehicle in order to calculate the road friction coefficient from the conditions of the vehicle while the vehicle is moving.
There are many other exemplary applications in various fields. What is common to those applications is that multiple linear regression analysis used as means for estimating parameters of a function from an input vector (explanatory variable) to an output value (objective variable) with a high degree of accuracy plays the primary role.
An equation used for the estimation can be represented as follows, for example, as described on page 165 of the article by K. Kachigan entitled “Multivariate Statistical Analysis”, Radius (1991). First, the equation (5) is transformed to
                              Y          =                                    θ              T                        ⁢            X                          ⁢                                  ⁢        where                            (        6        )                                X        =                                                                      1                                            1                                            ⋯                                            1                                                                                      x                  ⁢                                                                          ⁢                  1                  ⁢                                      (                    1                    )                                                                                                x                  ⁢                                                                          ⁢                  2                  ⁢                                      (                    1                    )                                                                              ⋯                                                              xN                  ⁡                                      (                    1                    )                                                                                                                        x                  ⁢                                                                          ⁢                  1                  ⁢                                      (                    2                    )                                                                                                x                  ⁢                                                                          ⁢                  2                  ⁢                                      (                    2                    )                                                                              ⋯                                                              xN                  ⁡                                      (                    2                    )                                                                                                      ⋯                                            ⋯                                            ⋯                                            ⋯                                                                                      x                  ⁢                                                                          ⁢                  1                  ⁢                                      (                    D                    )                                                                                                x                  ⁢                                                                          ⁢                  2                  ⁢                                      (                    D                    )                                                                              ⋯                                                              xN                  ⁡                                      (                    D                    )                                                                                                                    (        7        )                                θ        =                              (                          θ2              ,                              θ1                ⁡                                  (                  1                  )                                            ,                              θ1                ⁢                                                                  ⁢                                  (                  2                  )                                            ,              …              ⁢                                                          ,                              θ1                ⁡                                  (                  D                  )                                                      )                    T                                    (        8        )            
The equation for estimating parameter θ in the equation (8) is given asθ=(XXT)−1XYT  (9)
However, it is known that if the correlation between two components of a sample vector is strong, the matrix XXT approaches singularity and the accuracy of the parameter vector that can be obtained in accordance with the equation (9) degrades.
As techniques for preventing the degradation, Principle Component Regression (PCR) and Partial Least Square methods are disclosed in W. Wu and R, Manne: “Fast regression method in a Lanczos (or PLS-1) basis. Theory and applications”, Chemometrics and intelligent laboratory systems, 51, pp. 145-161 (2000) and R. Ergon: “Informative PLS score-loading plots for process understanding and monitoring”, Journal of Process Control, 14, pp. 889-897 (2004). These techniques use analysis of the principle component, select a base of a partial space that maximizes the distribution of X, and perform regression analysis based on the base. In particular, an equation for estimating a parameter is represented asθ=VS−1UTYT  (10)where V, S, and U are matrixes that can be obtained by singular value decomposition of X.X=USVT  (11)
If the correlation between the components of a sample vector is weak, a regression parameter can be estimated by using the equation (9) given above. If there are components having a strong correlation with each other, a regression parameter can be estimated by using the equation (10).
However, values estimated by using the parameter estimating method based on the least square method do not have consistency. That is, it is known that there remains an error between a parameter estimated by using the equation (9) or (10) and the true value no matter how many samples are used.
On the other hand, the article by Amari and Kawanabe entitled “Estimation of linear relations: Is the least square method the best?” Industrial and Applied Mathematics, Vol. 6, No. 2, pp. 96-109 (June 1996) discloses a new method for estimating parameters in which an evaluation function for parameter estimation for single linear regression analysis modeled by an equation (12) given below is represented by an equation (13).y=θ1x  (12)L=(Y−θ1TX)(Y−θ1TX)T/(1+θ12)  (13)
By applying a partial differential to the equation (13) with parameter θ1, the parameter estimation equationθ1={−(Sxx−Syy)±((Sxx−Syy)2+4Sxy)1/2}/(2Sxy)  (14)
can be obtained.
The appropriate one of the two solutions to the quadratic equation (14), for example the one that has a smaller estimation error, may be selected as the estimated parameter value. Estimated values obtained by using this parameter estimation method have consistency.
However, the conventional technique has the following problems.
The parameter estimation method based on the evaluation function in the equation (13) disclosed in the article by Amari and Kawanabe given above is nothing more than a method that uses the single linear regression model represented by the equation (14) that does not include a shift term. The article does not consider a single linear regression model including a shift term or an evaluation function and a parameter estimation method for general multiple linear regression model.